

<!DOCTYPE html>
<!--[if IE 8]><html class="no-js lt-ie9" lang="en" > <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en" > <!--<![endif]-->
<head>
  <meta charset="utf-8">
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  
  <title>Custom demos &mdash; Cortex2.0  documentation</title>
  

  
  
  
  

  

  
  
    

  

  <link rel="stylesheet" href="_static/css/theme.css" type="text/css" />
  <link rel="stylesheet" href="_static/pygments.css" type="text/css" />
    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" />
    <link rel="prev" title="Develop" href="develop.html" /> 

  
  <script src="_static/js/modernizr.min.js"></script>

</head>

<body class="wy-body-for-nav">

   
  <div class="wy-grid-for-nav">

    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search">
          

          
            <a href="index.html" class="icon icon-home"> Cortex2.0
          

          
          </a>

          
            
            
          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <p class="caption"><span class="caption-text">User Documentation</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="install.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="getting_started.html">Getting Started</a></li>
<li class="toctree-l1"><a class="reference internal" href="modules.html">cortex</a></li>
<li class="toctree-l1"><a class="reference internal" href="develop.html">Develop</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Custom demos</a></li>
<li class="toctree-l1"><a class="reference internal" href="#a-walkthrough-a-custom-classifier">A walkthrough a custom classifier:</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#building-models">Building models</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="#defining-losses-and-results">Defining losses and results</a></li>
<li class="toctree-l1"><a class="reference internal" href="#visualization">Visualization</a></li>
<li class="toctree-l1"><a class="reference internal" href="#putting-it-together">Putting it together</a></li>
</ul>

            
          
        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="index.html">Cortex2.0</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="index.html">Docs</a> &raquo;</li>
        
      <li>Custom demos</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
            
            <a href="_sources/build.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>

  
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
  <div class="section" id="custom-demos">
<h1>Custom demos<a class="headerlink" href="#custom-demos" title="Permalink to this headline">¶</a></h1>
<p>While cortex has built-in functionality, but it is meant to meant to be
used with your own modules. An example of making a model that works with
cortex can be found at:
<a class="reference external" href="https://github.com/rdevon/cortex/blob/master/demos/demo_classifier.py">https://github.com/rdevon/cortex/blob/master/demos/demo_classifier.py</a>
and <a class="reference external" href="https://github.com/rdevon/cortex/blob/master/demos/demo_custom_ae.py">https://github.com/rdevon/cortex/blob/master/demos/demo_custom_ae.py</a></p>
<p>Documentation on the API can be found here:
<a class="reference external" href="https://github.com/rdevon/cortex/blob/master/cortex/plugins.py">https://github.com/rdevon/cortex/blob/master/cortex/plugins.py</a></p>
<p>For instance, the demo autoencoder can be used as:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="n">cortex</span><span class="o">/</span><span class="n">demos</span><span class="o">/</span><span class="n">demo_custom_ae</span><span class="o">.</span><span class="n">py</span> <span class="o">--</span><span class="n">help</span>
</pre></div>
</div>
</div>
<div class="section" id="a-walkthrough-a-custom-classifier">
<h1>A walkthrough a custom classifier:<a class="headerlink" href="#a-walkthrough-a-custom-classifier" title="Permalink to this headline">¶</a></h1>
<p>Let’s look a little more closely at the autoencoder demo above to see
what’s going on. cortex relies on using and overriding methods of
plugins classes.</p>
<p>First, let’s look at the methods, <code class="docutils literal notranslate"><span class="pre">build</span></code>, <code class="docutils literal notranslate"><span class="pre">routine</span></code>, and
<code class="docutils literal notranslate"><span class="pre">visualize</span></code>. These are special methods for the plugin that can be
overridden to change the behavior of your model for your needs.</p>
<p>The signature of these functions look like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">build</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dim_z</span><span class="o">=</span><span class="mi">64</span><span class="p">,</span> <span class="n">dim_encoder_out</span><span class="o">=</span><span class="mi">64</span><span class="p">):</span>
    <span class="o">...</span>

<span class="k">def</span> <span class="nf">routine</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">inputs</span><span class="p">,</span> <span class="n">targets</span><span class="p">,</span> <span class="n">ae_criterion</span><span class="o">=</span><span class="n">F</span><span class="o">.</span><span class="n">mse_loss</span><span class="p">):</span>
    <span class="o">...</span>

<span class="k">def</span> <span class="nf">visualize</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">inputs</span><span class="p">,</span> <span class="n">targets</span><span class="p">):</span>
    <span class="o">...</span>
</pre></div>
</div>
<p>Each of these functions have arguments and keyword arguments. Note that
the keyword arguments showed up in the help in the above example. This
is part of the functionality of cortex: it manages your hyperparameters
to these functions, organizes them, and provides command line control
automatically. Even the docstrings are used in the command line, so
other users can get the usage docs directly from there.</p>
<p>The arguments are <em>data</em>, which are to be manipulated as needed in those
methods. These are for the most part handled automatically, but all of
these methods can be used as normal functions as well.</p>
<div class="section" id="building-models">
<h2>Building models<a class="headerlink" href="#building-models" title="Permalink to this headline">¶</a></h2>
<p>The <code class="docutils literal notranslate"><span class="pre">build</span></code> function takes the hyperparameters and sets networks.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Autoencoder</span><span class="p">(</span><span class="n">nn</span><span class="o">.</span><span class="n">Module</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">encoder</span><span class="p">,</span> <span class="n">decoder</span><span class="p">):</span>
        <span class="nb">super</span><span class="p">(</span><span class="n">Autoencoder</span><span class="p">,</span> <span class="bp">self</span><span class="p">)</span><span class="o">.</span><span class="fm">__init__</span><span class="p">()</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">encoder</span> <span class="o">=</span> <span class="n">encoder</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">decoder</span> <span class="o">=</span> <span class="n">decoder</span>

    <span class="k">def</span> <span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">,</span> <span class="n">nonlinearity</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
        <span class="n">encoded</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">encoder</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
        <span class="n">decoded</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">decoder</span><span class="p">(</span><span class="n">encoded</span><span class="p">)</span>
        <span class="k">return</span> <span class="n">decoded</span>

<span class="o">...</span>

    <span class="k">def</span> <span class="nf">build</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">dim_z</span><span class="o">=</span><span class="mi">64</span><span class="p">,</span> <span class="n">dim_encoder_out</span><span class="o">=</span><span class="mi">64</span><span class="p">):</span>
        <span class="n">encoder</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Sequential</span><span class="p">(</span>
            <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">28</span><span class="p">,</span> <span class="mi">256</span><span class="p">),</span>
            <span class="n">nn</span><span class="o">.</span><span class="n">ReLU</span><span class="p">(</span><span class="kc">True</span><span class="p">),</span>
            <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">256</span><span class="p">,</span> <span class="mi">28</span><span class="p">),</span>
            <span class="n">nn</span><span class="o">.</span><span class="n">ReLU</span><span class="p">(</span><span class="kc">True</span><span class="p">))</span>
        <span class="n">decoder</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Sequential</span><span class="p">(</span>
            <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">28</span><span class="p">,</span> <span class="mi">256</span><span class="p">),</span>
            <span class="n">nn</span><span class="o">.</span><span class="n">ReLU</span><span class="p">(</span><span class="kc">True</span><span class="p">),</span>
            <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">256</span><span class="p">,</span> <span class="mi">28</span><span class="p">),</span>
            <span class="n">nn</span><span class="o">.</span><span class="n">Sigmoid</span><span class="p">())</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">nets</span><span class="o">.</span><span class="n">ae</span> <span class="o">=</span> <span class="n">Autoencoder</span><span class="p">(</span><span class="n">encoder</span><span class="p">,</span> <span class="n">decoder</span><span class="p">)</span>
</pre></div>
</div>
<p>All that’s being done here is the hyperparameters are being used to
create an instance of an <code class="docutils literal notranslate"><span class="pre">nn.Module</span></code> subclass, which is being added to
the set of “nets”. Note that they keyword <code class="docutils literal notranslate"><span class="pre">ae</span></code> is very important, as
this is going to be how you retrieve your nets and define their losses
farther down.</p>
<p>Also note that cortex <em>only</em> currently supports <code class="docutils literal notranslate"><span class="pre">nn.Module</span></code> subclasses
from Pytorch.</p>
</div>
</div>
<div class="section" id="defining-losses-and-results">
<h1>Defining losses and results<a class="headerlink" href="#defining-losses-and-results" title="Permalink to this headline">¶</a></h1>
<p>Adding losses and results from your model is easy, just compute your
graph given you models and data, then add the losses and results by
setting those members:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">routine</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">inputs</span><span class="p">,</span> <span class="n">targets</span><span class="p">,</span> <span class="n">ae_criterion</span><span class="o">=</span><span class="n">F</span><span class="o">.</span><span class="n">mse_loss</span><span class="p">):</span>
    <span class="n">encoded</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">nets</span><span class="o">.</span><span class="n">ae</span><span class="o">.</span><span class="n">encoder</span><span class="p">(</span><span class="n">inputs</span><span class="p">)</span>
    <span class="n">outputs</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">nets</span><span class="o">.</span><span class="n">ae</span><span class="o">.</span><span class="n">decoder</span><span class="p">(</span><span class="n">encoded</span><span class="p">)</span>
    <span class="n">r_loss</span> <span class="o">=</span> <span class="n">ae_criterion</span><span class="p">(</span>
        <span class="n">outputs</span><span class="p">,</span> <span class="n">inputs</span><span class="p">,</span> <span class="n">size_average</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span> <span class="o">/</span> <span class="n">inputs</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">losses</span><span class="o">.</span><span class="n">ae</span> <span class="o">=</span> <span class="n">r_loss</span>
</pre></div>
</div>
<p>Additional results can be added similarly. For instance, in the demo
classifier:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">routine</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">inputs</span><span class="p">,</span> <span class="n">targets</span><span class="p">,</span> <span class="n">criterion</span><span class="o">=</span><span class="n">nn</span><span class="o">.</span><span class="n">CrossEntropyLoss</span><span class="p">()):</span>
    <span class="o">...</span>
    <span class="n">classifier</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">nets</span><span class="o">.</span><span class="n">classifier</span>

    <span class="n">outputs</span> <span class="o">=</span> <span class="n">classifier</span><span class="p">(</span><span class="n">inputs</span><span class="p">)</span>
    <span class="n">predicted</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="n">F</span><span class="o">.</span><span class="n">log_softmax</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">dim</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="mi">1</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span>

    <span class="n">loss</span> <span class="o">=</span> <span class="n">criterion</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">targets</span><span class="p">)</span>
    <span class="n">correct</span> <span class="o">=</span> <span class="mf">100.</span> <span class="o">*</span> <span class="n">predicted</span><span class="o">.</span><span class="n">eq</span><span class="p">(</span>
        <span class="n">targets</span><span class="o">.</span><span class="n">data</span><span class="p">)</span><span class="o">.</span><span class="n">cpu</span><span class="p">()</span><span class="o">.</span><span class="n">sum</span><span class="p">()</span> <span class="o">/</span> <span class="n">targets</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>

    <span class="bp">self</span><span class="o">.</span><span class="n">losses</span><span class="o">.</span><span class="n">classifier</span> <span class="o">=</span> <span class="n">loss</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">results</span><span class="o">.</span><span class="n">accuracy</span> <span class="o">=</span> <span class="n">correct</span>
</pre></div>
</div>
</div>
<div class="section" id="visualization">
<h1>Visualization<a class="headerlink" href="#visualization" title="Permalink to this headline">¶</a></h1>
<p>Cortex allows for visualization using visdom, and this can be defined in
a similar way as above:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">visualize</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">images</span><span class="p">,</span> <span class="n">inputs</span><span class="p">,</span> <span class="n">targets</span><span class="p">):</span>
    <span class="n">predicted</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">inputs</span><span class="p">)</span>
    <span class="bp">self</span><span class="o">.</span><span class="n">add_image</span><span class="p">(</span><span class="n">images</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="n">labels</span><span class="o">=</span><span class="p">(</span><span class="n">targets</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="n">predicted</span><span class="o">.</span><span class="n">data</span><span class="p">),</span>
                   <span class="n">name</span><span class="o">=</span><span class="s1">&#39;gt_pred&#39;</span><span class="p">)</span>
</pre></div>
</div>
<p>See the ModelPlugin API for more more details.</p>
</div>
<div class="section" id="putting-it-together">
<h1>Putting it together<a class="headerlink" href="#putting-it-together" title="Permalink to this headline">¶</a></h1>
<p>Finally, we can specify default arguments:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">defaults</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span>
    <span class="n">data</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span>
        <span class="n">batch_size</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">train</span><span class="o">=</span><span class="mi">64</span><span class="p">,</span> <span class="n">test</span><span class="o">=</span><span class="mi">64</span><span class="p">),</span> <span class="n">inputs</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">inputs</span><span class="o">=</span><span class="s1">&#39;images&#39;</span><span class="p">)),</span>
    <span class="n">optimizer</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">optimizer</span><span class="o">=</span><span class="s1">&#39;Adam&#39;</span><span class="p">,</span> <span class="n">learning_rate</span><span class="o">=</span><span class="mf">1e-4</span><span class="p">),</span>
    <span class="n">train</span><span class="o">=</span><span class="nb">dict</span><span class="p">(</span><span class="n">save_on_lowest</span><span class="o">=</span><span class="s1">&#39;losses.ae&#39;</span><span class="p">))</span>
</pre></div>
</div>
<p>and then add <code class="docutils literal notranslate"><span class="pre">cortex.main.run</span></code> to <code class="docutils literal notranslate"><span class="pre">__main__</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">autoencoder</span> <span class="o">=</span> <span class="n">AE</span><span class="p">()</span>
    <span class="n">run</span><span class="p">(</span><span class="n">model</span><span class="o">=</span><span class="n">autoencoder</span><span class="p">)</span>
</pre></div>
</div>
<p>And that’s it. cortex also allows for lower-level functions to be
overridden (e.g., train_step, eval_step, train_loop, etc) with more
customizability coming soon. For more examples of usage, see the
built-in models:
<a class="reference external" href="https://github.com/rdevon/cortex/tree/master/cortex/built_ins/models">https://github.com/rdevon/cortex/tree/master/cortex/built_ins/models</a></p>
</div>


           </div>
           
          </div>
          <footer>
  
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
      
      
        <a href="develop.html" class="btn btn-neutral" title="Develop" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left"></span> Previous</a>
      
    </div>
  

  <hr/>

  <div role="contentinfo">
    <p>
        &copy; Copyright 2018, MILA.

    </p>
  </div>
  Built with <a href="http://sphinx-doc.org/">Sphinx</a> using a <a href="https://github.com/rtfd/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>. 

</footer>

        </div>
      </div>

    </section>

  </div>
  


  

    <script type="text/javascript">
        var DOCUMENTATION_OPTIONS = {
            URL_ROOT:'./',
            VERSION:'',
            LANGUAGE:'None',
            COLLAPSE_INDEX:false,
            FILE_SUFFIX:'.html',
            HAS_SOURCE:  true,
            SOURCELINK_SUFFIX: '.txt'
        };
    </script>
      <script type="text/javascript" src="_static/jquery.js"></script>
      <script type="text/javascript" src="_static/underscore.js"></script>
      <script type="text/javascript" src="_static/doctools.js"></script>
      <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>

  

  <script type="text/javascript" src="_static/js/theme.js"></script>

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script> 

</body>
</html>